Empirical Policy Evaluation With Supergraphs

We devise algorithms for the policy evaluation problem in reinforcement learning, assuming access to a simulator and certain side information called supergraph. Our explore backward from high-cost states find high-value ones, contrast approaches that work forward all states. While several papers have demonstrated utility of exploration empirically, we conduct rigorous analyses which show our can reduce average-case sample complexity O(S logS) as low O(logS). Analytically, adapt tools network science literature provide new methodology learning problems.